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(54) Change log aggregation and optimization 

(57) A change log aggregation and optimization 
mechanism and methodology for updating and synchro- 
nizing application data and application files In a client 
device of a data transfer and synchronization system. 
The contents of a plurality of change logs reflecting the 
then cun-ent changes to the application data of the client 
device are downloaded and merged into an aggregate 
log. Instead of applying each change log as it is down- 
loaded, the contents of the aggregate log, representing 
all changes to application data and/or application files 
recorded in prior change log^, are then applied to the 
client device to update application data and/or applica- 
tion files in the client device. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[(X)01] The invention relates to the transference of da- 
ta between two systenrts independent of the form in 
which the data is Icept on the respective systems, and 
in particular to providing an efficient means of commu- 
nicating data between systems and devices. 

Description of the Related Art 

[0002] The growth of computing-related devices has 
not been limited to personal computers or workstations. 
The number of personal computing devices has grown 
substantially in both type and format. Small, hand-held 
computers carry a multitude of contact, personal, docu- 
ment, and other infonnation and are sophisticated 
enough to allow a user to fax, send e-mails, and com- 
municate in other ways wirelessly. Even advanced cel- 
lular phones carry enough memory and processing 
power to store contact infonnation, surf the web, and 
provide text messaging. Along with the growth in the so- 
phistication of these devices, the need to transfer infor- 
mation between them has grown significantiy as well. 
[0003] With a multitude of different device types on 
the market, Iceeping information between the different 
devices synchronized has become increasinoty prob- 
lematic. For example, If an Individual keeps a calendar 
of information on a personal computer in his or her office 
using a particular personal information manager appli- 
cation, the individual would generally like to have the 
same Infonnation available in a cellular phone, hand- 
held organizer, and perhaps a home personal computer. 
The individual may additionally have a notebook com- 
puter which requires synchronizing file data such as 
presentations or working documents between the note- 
book and the office computer. 
[0O04] Until now, synchronization between both doc- 
uments and personal information managers has oc- 
curred through direct connection between the devices, 
and generally directly between applications such as a 
personal infonnation manager In one device and a per- 
sonal infonnation manager in another device or using 
an intemrtediary svnc-mappinq oroaram. 
[0005] One example of this is the prevalent use of the 
3Com Palm® OS-based organizer, such as the 3Com 
Palm® series of computing devices, which uses its own 
calendaring system, yet lets users synchronize the data 
therein with a variety of different personal infonnation 
manager software packages, such as Symantec's ACTI 
^**, Microsoft's Outlook®, and other systems. In this ex- 
ample, an intermediary synchronization program such 
as Puma Technology, Inc.'s Intellisync® is required. In- 
tellisync® is an application program whicn mns on both 
the hand-held device and the computer which stores the 
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information data and maps data systems between non- 
uniform data records. 

[0006] In other cases, direct transfer between appli- 
cations such as transfer between Microsoft's Outlook® 
computer-based client and Microsoft's Windov^ CE 
•Pocket Outlook* application, is possible. Nevertheless, 
in both cases, synchronization occurs through direct 
connection between a personal computer and the oer- 
sonal computing device, While this connection is gen- 
erally via a cable directly connecting, for example, a 
Palm® devfee in a cradle to the personal computer, the 
connection nnay be wireless as welt. 
[0007] One component of these synchronization sys- 
tems is that the synchronization process must be able 
to delineate between when changes are made to spe- 
cific databases and must make a decision about wheth- 
er to replace the changed field. Normally, this is meas- 
ured by a change in one database, and no-change in a 
second database. In some cases, both databases will 
have changed between syncs. In this case, the sync op- 
eration must determine which of the two changes, whfch 
has been made is to 'win* and replace the other during 
the sync. Generally, this determinant of whether a con- 
flict exists allows some means for letting the user re- 
solve the confltet. 

[0008] In a technical sense, synchronization in this 
manner is generally accomplished by the copying of full 
records between systems. At some level, a user is gen- 
erally required to map data fields from one application 
to flnoth<*r f»nr* ppec^ w>>»rh <^=i*9 ^^eW*? ?.r<? ^ss'ir'r^ocJ 
which corresponding field in a different devk^e. Less 
mapping is required where developers more rol>ustly 
support various platforms of applications. 
[0009] In many instances, the data to be synchronized 
is generally in the form of text data such as records of 
addresses, contact information, calendar information, 
notes and other types of contact information. In certain 
instances, data, to be synchronized will be binary format 
of executable files or word processor-specifk: docu- 
ments. In many cases where document synchronization 
is required, the synchronization routine simply deter- 
mines whether or not the documents in question have 
changed, and uses a time-based representation to de- 
termine which of the two files is newer j?nd replaces the 
older file with the newer file to achieve synchronization, 
as long as the older of the two files was in fad not 
chanqed. This is the morfel Msed »n the ff?m!ll9r "Brief- 
case" function in Mk:rosoft Windows-based systems. If 
both files have changed, then the synchronization rou- 
tine presents the option of conflict resolution to the user. 
[0010] Generally, such synchronization schemes are 
relatively inefficient since they require full bandwidth of 
the document or binary file to be transfen-ed via the syn- 
chronization link. In addition, a change log is typk:ally 
generated with each synchronization operation to 
record the changes in any given data record. In a situ- 
ation wnere a large number of change logs have been 
generated (and stored in a server's memory), the se- 
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quence of downloading and applying each change log 
often results in the unnecessary download of data. A 
separate transaction is required for each occun'ence of 
an Item record in the change log sequence and it is pos- 
sible for the same field in a record to be updated many s 
times during a single syncfironization operation. 
[0011] Accordingly, there Is a need for a change log 
aggregation and optimization mechanism that will more 

the synchronization process. io 

SUMMARY OF THE INVENTION 



an apparatus for updating the application data in a client 
device of a data trartsfer and synchronization system. 
The apparatus may comprise: a downloading routine for 
(teratively retrieving a plurality of change logs from a 
server system where each of the plurality of change logs 
reflect the then cun-ent changes to the application data; 
a merging routine for iteratively aggregating the con- 
tents of the plurality of change lo^ to an aggregate 

deleting the plurality of change logs; and an updating 
routine for applying the contents of the aggregate log to 
the application data to update the application data. 



[0012] The invention, roughly described, provides a 
method and apparatus for merging the contents of a plu- 15 
ralrty of change logs into an aggregate log in a down- 
load-and-apply sequence of a client device to update 
application data in the client device of a data transfer 
and synchronization system. Redundant changes in 
multiple change logs are eliminated and files and 20 
records of the application data can thus be synchronized 
and updated more efficiently via a single transaction. 
[0013] In one aspect, the invention provides a method 
for updating application data in a client device of a data 
transfer which may include the steps of: downloading a 25 
first change log of a plurality of change logs from a serv- 
er system, where each of the plurality of change logs 
reflects changes to the application data; adding the first 
change log to an aggregate log; deleting the first change 
log; repeating the downloading, adding, and deleting so 
steps ior a nexi cnange (og ot tne plurality of cnange 
logs until no additional change logs exist; and applying 
the aggregate log to the application data to update the 
application data. 

[0014] The step of adding or merging the plurality of 35 
change logs to the aggregate log may further comprise 
the steps of: retrieving information for a valid item of the 
application data; updating a map of the aggregate log; 
writing the Item to the aggregate log; updating the loca- 
tion of the valid item in the map; and repeating the pre- 40 
vious steps for all remaining valid items of the cun-ent 
change log being rolled into the aggregate log. 
[0015] In a further aspect, the invention comprises a 
method for combining changes in a plurality of accumu- 
lated change ioga into an ayyrttga'ie iog. Tiie nielnoci 
may include the steps of: creating an aggregate log; re- 
trieving the contents of a current change log; adding the 
coiueiiLS ui ii'itfcurieiilcircttiijiB luv) lO 11 le dgijieydie lug, 
deleting the current change log; and repeating the pre- 
vious steps until no additional change logs exist. In par- so 
ticular, said step of adding or rolling the content of the 
change logs into the aggregate log may comprise the 
steps of: retrieving the contents of a plurality of items of 
the application data; updating a map of the items; up- 
dating the aggregate log; updating the location of re- 55 
spective items in the map; and compacting the aggre- 
gate fog if 9 compact threshold evceeded. 
[001 6] In a further aspect, the invention may comprise 



BRIEF DESCRIPTION OF THE DRAWINGS 

[001 7] The invention wilt be descn'bed with respect to 
the particular embodiments thereof. Other objects, fea- 
tures, and advantages of the invention will become ap- 
parent with reference to the specification and drawings 
in which: 

Figure 1 is a exemplar block diagram of a data 
transfer and synchronization system having a 
change log aggregation and optimization mecha- 
nism in accordance with the present invention. 
Figure 1 A is a flow diagram illustrating a start down- 
load and aggregate sequence of the present inven- 
tion. 

Figure 2 is a flow diagram illustrating the sequence 
of steps for adding a change log to an aggregate 
change tog in accordance with the pr^ent inven- 
tion. 

Figure 3 is a flow diagram illustrating the steps for 
retrieving items of a change log in accordance with 
the present invention. 

Figure 4 is a flow diagram illustrating the steps for 
updating a map of a change log in accordance with 
the present invention. 

Figure 5 is a flow diagram illustrating the sequence 
of steps for adding a file item in accordance with the 
present Invention. 

Rgure 6 is a flow diagram illustrating the default se- 
quence of steps for adding an item in accordance 
with the present invention. 
Figure 7 is a liow diagram illustrating the steps for 
merging fields of an item in accordance with the 
present invention. 

rigute o is a ilow aiagram niusirating tne steps for 
merging fields of an collection in accordance with 
the present Invention. 

Figure 9 is a flow diagram illustrating the steps for 
compacting an aggregate log in accordance with 
the present invention. 

Figure 10 is a flow diagram illustrating the steps for 
retrieving fields from a current item of a change log 
in accordance with the present Invention. 
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• DETAILED DESCRIPTION 

[001 8] Rg. 1 is a generalized block diagram of an ex- 
emplary data transfer and synchronization system. In 
particular, the system includes network 1 0 that includes s 
one or more storage servers 1 2, 14 and a management 
server 15. A plurality of client devces (e.g., home PC 
1 6, office PC 1 8, personal Infomnation PaM^computing 
device ^u, and weoenaoiea ceiiuiar ceiepnone are 
capable of coupling to network 1 0 and exchanging syn- io 
chronization information in an offline fashion via the stor- 
age servers 12 and 14, in accordance with the tech- 
niques described in U.S. Patent Application Serial No. 
09/490.550, filed January 25. 2000, U.S. Patent Appli- 
cation Serial No. 09/491 ,675 and U.S. Patent Applk:a- is 
tion Serial No. 09/491 ,694 both Hied January 26, 2000, 
and all entitled 'Data Transfer and Synchronization Sys- 
tem,' incorporated herein by reference. 
[0019] Client software of the present invention is in- 
stalled on both home PC 1 6 and office PC 1 8 of Fig. 1 , 20 
for example, and is configured to operate in conjunction 
with an operating system such as Mk^rosoft Windows®. 
The client software, when executed, Interacts with the 
various applications on the user's PCs. The user inter- 
acts with the client software and configures the software 2s 
such that the appfications are prioritized. Data is then 
extracted from the various applbatlons, organized in a 
format independent of the particular application and de- 
vice from which the data originated, and incorporated 
into a data package. With exemplary embodiments of so 
the present invention, various classes of data, Including 
contacts (e.g., names, addresses, phone nunrA}ers, 
email addresses), Intemet browser bookmarks (e.g., 
Netscape Navigatoi®. Microsoft Explorei®), calendar 
events, email messages (e.g., Inbox, Sent Items, Delet- 33 
ed Itenns), notes, tasks, and files (e.g., word processor 
specific documents, electronic presentations, spread- 
sheets, binary format of executable files) can be manip- 
ulated and synchronized between the plurality of devic- 
es. 40 
[0020] Consider, for example, a scenario where a us- 
er has installed the Mk;rosoft Outlook® application on 
home PC 1 6 and the user has entered the infonnation 
of five individuals into the Contacts folder. When in- 
structed by the user to synchronize the contents of the ^5 
Contacts folder, the client software accesses Outiook® 
and assembles the infonnation for five individuals in a 
DataPack CONT.DOOO. In this case, the UUID "CONT* 
identifies Contacts information as the spedfic object and 
suffix DOOO signifies that this DataPack is version '0." so 
The contacts are combined in DataPack CONT.DOOO as 
a collection of five transactions, each of which is as- 
signed a unique ID number 1 . 2, ... 5. ID^ may represent 
contact information for Jane Smith and ID2 may repre- 
sent contact information for John Doe. In this example, ss 
each transaction 1 , 2. 5 is associated with an associ- 
aled dClion, "Add." DalaPack CONTDOOO is subse- 
quently uploaded to network 10 and stored on, for ex- 
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ample, storage server 12. 

[0021 ] Later, office PC 1 8 connects to networic 1 0 and 
office PC 18 sends a signal to management server 15 
infomning management server 15 that office PC 18 has 
not downloaded any DataPacks. Management server 
1 5 responds by sending a signal to office PC 1 8 indicat- 
ing that a DataPack CONT.DOOO of change Information 
for contacts has been stored on storage server 12 since 
omce 1 0 lasi conneciea to netwonc 1 u. umce 1 », 
in response, sends a signal to management server 15 
requesting DataPack CONT.DOOO. The most recent da- 
ta package(s) stored on storage server 12 are identified 
(in this example, version 0 of the contact data, CONT. 
DOOO) and DataPack CONT.DOOO is then downloaded 
to office PC 18. 

[0022] The change infonnation in that data package, 
"Adds" of contacts In this case, is then applied to the 
pertinent application in office PC 18. In this example, the 
client software on the offrce PC is configured to synchro- 
nize contacts using a Lotus Notes® application. Thus, 
the five Adds from CONT.DOOO are applied to the con- 
tacts in Lotus Notes(&. so that the contact information in 
home PC 16 and offk:e PC 18 is synchronized. The of- 
fice PC 18 then sends a signal back to management 
server 15 indicating that offk:e PC 18 has applied ver- 
sion "0" of the contacts data package(s). This informa- 
tion is preferably maintained in a registry by manage- 
ment server 15 for each and every devk^e that couples 
to the network to download and upload change informa- 
tion. 

[0023] Sut>sequently, the user of office PC 1 6 updates 
the contacts in Lotus Notes® and adds one or more con- 
tacts. In this example, ten new contacts are added. 
Thus, the Lotus Notes® application uploads a second 
data package to networic 1 0, the data package including 
the ten contacts and each having the associated action, 
"Add." This data package represents more recent 
change information than the information in CONT.DOOO. 
The data package uploaded by offk:e PC 1 8 is Identified 
by a unique filename, in this example. CONff.DOOl . In 
addition, office PC 1 8 sends a signal to management 
server 15 confirming that CONT.D001 has been upload- 
ed to network 1 0. The registry is updated to indicate that 
office PC 1 8 has already applied the change Information 
inCONT.DOOI. 

[0024] Home PC 16 subsequently connects to data 
network 1 0, and the client software on home PC 1 6 com- 
municates with management server 15 coupled to net- 
work 10. In particular, home PC 16 sends a signal to 
management server 15 identifying CONT. DOOO as the 
most recent version of change information the last time 
home PC 1 6 was coupled to data network 1 0. Manage- 
ment server 1 5 queries the storage servers for any more 
recent data packages of changes to contact information. 
Management server 1 5 identifies such data package(s). 
CONTD001 In this example, and sends a signal to home 
PC 16 iiirorming hotne PC 16 Liiat sucii dala package 
(s) exist. The client software on home PC 1 6 then re- 
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1 quests the new data paclcage(s), and management 
server 15 then downloads the data package(s) to home 
PC 15. The change infomtation therein, in this case the 
ten contacts from COI^JT.0001 to be added, ts then ap- 
plied to the Contact information maintained by Microsoft 5 
Outlook® on home PC 1 6. The communication of the 
change information to Microsoft Outlook® and sut)se- 
quent updates to the contacts in Outlook® are coordi- 

Contact information in home PC 16 and work PC 18 is io 
again synchronized. 

[0025] Other transactions, in addition to "Add/ are 
provided with exemplary embodiments of the present in- 
vention. One of these is the transaction, "Modify.* Using 
the present example, the contact infonnation for a per- ^5 
son may sometimes change after CONT.D001 is up- 
loaded to network 1 0. For instance. Jane Smith may call 
the user on the telephone and tells the user that Jane 
has changed her phone number. The user then access- 
es office PC 18 and changes the phone number for Jane 20 
Smith in the user's Contacts folder. 
[0026] The user then activates, for example, a "Syn- 
chronize" button displayed on the computer screen by 
the client software, so a new data package or the 
change log, CONT.D002, is created and uploaded to 25 
network 1 0 and stored on one of storage senders 12, 14. 
A signal is sent by office PC 1 8 to management server 

15 infomning managennent server 15 that data package 
COrsJT.D002 has been uploaded. Data package CONT. 
D002 differs from data packages CONTDOOO and 30 
CONT.D001 , in that the action, "Modify" is used instead 

of "Add." The Modify instruction and the associated 
change infonnation is conflated with the particular user. 
In particular, the Modify instruction is associated with the 
pertinent ID, in this case 10^ representing Jane Smith. 35 
In addition, data package CONT.D002 includes the field 
to be modified, in this example, "Phone," and the new 
Information, in this exannple, Jane Smith's new phone 
number. 

[0027] Subsequently, when home PC 1 6 connects to 40 
network 1 0, the data package CONT.0002 is download- 
ed to home PC 1 8, and the client software recognizes 
that, for IDi the infonmatlon within the field "Phone" has 
been updated. The modification Is then made to this 
contact information via Microsori uutiooKK^. Home PC ^ 

16 then sends a confinTtation signal to management 
server 1 5, conf inning that home PC 1 6 has received and 
appiiea tne cnange inronnnation in version tr^ or tne con- 
tacts data packages. The pertinent information In the 
registry for home PC 1 6 is then updated. The next time so 
home PC 1 6 couples to network 1 0, home PC 1 6 sends 

a signal to management server indicating that home PC 
16 has received CONT0002. If no subsequent data 
packages with change information for contacts have 
been stored on the storage servers, then no data pack- 55 
ages are downloaded to home PC 16. 
[0023] char.scs arc rr.sd2 fcr various classes g* 
data, data packages accumulate on storage servers 12, 



1 4 and consume valuable storage space. As the nunnber 
of stored data packages increases, the amount of avail- 
able storage space on storage servers 12, 14 con^- 
spondingly decreases. In the example above, assume 
for illustrative purposes that data package COf^.DOOO 
occupies 2 kilobytes ("K") of memory on storage server 
12, COf^.DOOl occupies 1 K, and COfsrr.D002 occu- 
pies 0.5 K, for a total of 3.5 K memory. In sltuatk)ns a 

uaci K> liiiiiicu, lui c:Aaiii}jic, lu 25 iiieijiiuyied ("m ') ui* 

memory, the amount of available storage space contin- 
ues to decrease as data packages are updated and 
change logs are accumulated, until storage space on 
the storage servers 12, 14 are exhausted. 
[0029] Thus, prior to the present Invention, each syn- 
chronization created a unique change log to record any 
changes In the data to be synchronized. The change log 
Is subsequently uploaded to network 10 via, for exam- 
ple, an Internet connection and stored in storage server 
12. Overtime, as the user performs additional synchro- 
nizations, the change logs accumulate and when a new 
client devbe, or a client device that has not been recent- 
ly synchronized, is connected to network 1 0 and syn- 
chronized, all or some substantial subset of the accu- 
mulated change logs are downloaded and applied one 
at a time. Thus, if there are three change logs (e.g., a 
first change log may record a change in a contacts 
home phone number, a second change log may record 
an addition of a work phone number for the contact, and 
a third change log may yet record the deletion of the 
contact entirely), ail three change logs must be down- 
loaded and applied to the client devce one at a time 
when a new client device is subsequently synchronized, 
even though the contact is ultffnately deleted. 
[0030] In situations where there Is a sequence of 
change logs, there may be numerous conflbting chang- 
es to the same item. When this happens the user is 
asked to decide which of the conflk:ting versions to 
keep. Prior to change log aggregation (I.e.. with multiple 
change logs), each time an Item, for which a conflct ex- 
isted, was changed, the user was required to resolve 
ostensibly the same conflict. In an aggregated log, there 
would generally be only one entry for each item and 
thus, thus only one possible conflict to resolve. 
[0031] Also, In situations where a large number of 
change logs have been generated, there ts considerable 
potential inefficiency inherent in the old sequence of 
downloading and applying each change log In tum. In a 
given item entry tnere may only be one or two fields and 
there may be other fields belonging to the same Item 
scattered throughout entries In the sequence of change 
logs. Moreover, some of these later entries overwrite 
earlier ones. As a result, a separate transaction is re- 
quired for each occurrence of an item record In the 
change log sequence and it is possible for the same field 
in a record to be updated many times during a single 
sync. With an aggregated change log redundant chang- 
es zr2 ollmiratsd and records cen be upuated rr.crs ef- 
ficiently via a single transaction. To further maximize the 



5 



9 



EP1 180 890 A2 



10 



efficiency of the present invention, a means Is provided 
for compacting or compressing the aggregated change 
log rf a compact threshold is ever exceeded. 
[0032] For items consisting of large quantities of data 
such as files (e.g., Microsoft Word® document), chang- 
es are transmitted as "binary deltas." These are opti- 
mized packages, which encode the infomiation required 
to convert a binary image from one "version" to the next. 

Ao ^icu'ik^a awioutViuiciif; uic olcc ui u its ii luiviuuot uciLoa, 

taken together, eventually exceeds the size of the object 
itself. At this point a new base version, or version "0," 
may be established and the complete object sent How- 
ever, the change logs contain an identifier not the actual 
delta, or data. This identifier is used to request the delta 
itself, when the change log is applied. Thus, in aggre- 
gating change logs for files, if a new base version is en- 
countered existing changes can be flushed and only the 
new base and any subsequent changes need be down- 
loaded. Also, if a user has a conflict and resolves it by 
choosing a local version of the file, a new version "0" 
must be established, as there is no way of knowing the 
relationship between the local version and those in the 
stored change logs. 

[0033] The change log aggregation and optimization 
mechanism of the present Invention is implemented In 
the client devfce. Specifically, the change log aggrega- 
tion and optimization mechanism of the present inven- 
tion is inserted into the download-and-apply sequence 
where change logs are fetched and interpreted to up- 
date application data. Instead of applying each change 
log as it is downloaded, it is merged into the "rolled" or 
aggregate log. The change log aggregation and optimi- 
zation mechanisrh of the present invention is further il- 
lustrated in the flow diagrams of Figures 1 A and 2. 
[0034] Figure 1 A illustrates the addition of a cun^ent 
change log to an aggregate change log. The download 
and aggregate sequence begins in Step 100 of Figure 
1 A. A rolled log, or aggregate log, is created in Step 1 03. 
For each change log In the sequence, starting with Step 
106 the following steps are performed. In Step 109 the 
cun-ent change log is retrieved or downloaded. The cur- 
rent change log is then added to the aggregate log in 
Step 1 1 2. The current change log is then deleted in Step 
1 1 5 and Step 1 1 8 repeats Steps 1 09, 11 2, and 1 1 5 until 
there are no additional change logs in the sequence. 
Once this happens, the aggregate log is closed in step 
120 and the download and aggregate sequence is ter- 
mmaiea in biep ^z^, me atan Download and Aggre- 
gate sequence of the present invention can be summa- 
rized in the following steps: 

1 . Create aggregate log; 

2. Fetch (download) change log; 

3. Add change log to aggregate; 

4. Delete change log; 

5. Repeat the fetch add delete sequence for each 
change log to be £ggrag£t3d; and 

6. Close aggregate log. 



[0035] Rgure 2 shows an expanded view of Step 1 1 2 
of Rgure 1 A. For each item in the current change log 
the following steps are performed. The item in the cur- 
rent change log is first retrieved in Step 206 and the map 
5 is upcfeited in Step 209. The map stores meta-data to 
facilitate the aggregation or nnerging operation. It allows 
a one-to-many relationship between the keys and the 
itenns with which they are associated. This is a require- 
iiitriiLtui iitfiiiiypt^ wiiu^euimriges me reaii^ea as ^aei- 
tas," as each change must be applied separately and in 
order. Thereafter, in Step 212 the aggregate log is up- 
dated and in Step 215 the item location information is 
also updated. Simultaneously in Steps 21 8 and 221 the 
cun-ent item retrieved from the current change log is writ- 
ten to and stored in the aggregate log. In retrieving and 
merging subsequent change logs into the aggregate 
log, It is important to note that this process combines the 
data from two records for the same item using the meta- 
data in the map to locate each field and find its data in- 
side the con-ect change log. Once the item with its 
merged data has been written to the end of the aggre- 
gate log, the previous entry is marked obsolete, and can 
thus be deleted at a later time. 
[0036] Once there are no additional items in the cur- 
rent change log, in Step 227, the present Invention de- 
temiines whether the compact threshold has been ex- 
ceeded. If the compact threshold has been exceeded 
then, in Step 230, a compact aggregate log step 230 is 
Initiated to compress or compact the size of the aggre- 
gate log. Compact aggregate log step 230 essentially 
removes obsolete records in the aggregate log to mini- 
mize the size of the aggregate log and thus conserve 
memory in storage servers 12, 1 4. This is accomplished 
by iterating over each record in the aggregate log, read- 
ing only the valid records (having skipped over any ob- 
solete items), and writing the item back into the aggre- 
gate log without skipping obsolete items, effectively slid- 
ing data in the aggregate log back over obsolete records 
and thus eliminating them. 

[0037] Othenwfee if thecompactthreshold in Step227 
has not been exceeded then the sequence of steps in- 
itiated by Step 112 terminates In Step 233. The series 
of steps beginning with Step 112 effectively reads each 
record from the current change log and places this in- 
fonnation In the aggregate log. The Add Change Log to 
Aggregate Log sequence of Fig. 2 can be summarized 
in the following steps: 

1 . Get item information; 

2. Update the map; 

3. Update the aggregate log; 

4. Update item location information (in the map); 

5. Repeat the above sequence while we get a valid 
item; and 

6. Compact the aggregate log if the compact thresh- 
old has been reached. 

[0038] Figure 3 shows an expanded view of Step 206 
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of Figure 2. In Step 303, the present invention first de- 
termines whether the current position is less than the 
size of the change log. If the current position of the 
record is not less than the size of the change log then 
an invalid message is returned in Step 339 and the proc- 
ess terminates in Step 342. However if the cun-ent po- 
sition is less than the size of the change log then an 
operation code is read in Step 306. The present inven- 

Ws^tt incn ucici 1 1 til ica, iit olcpoOg, wiit^iiitft iiieupmaiiuH 

code is associated with a valid operation. If it is not a 
valid operation then a log error is generated in Step 31 8. 
an invalid message is retumed in Step 339 and the proc- 
ess temninates with Step 342. If, however, the operation 
code is associated with a valid operation the current po- 
sition of the record is retrieved from the change log in 
Step 31 2. In Step 31 5, the invention detennines whether 
the operation code con-esponds to a "NOP," or a non- 
operation, in Step 315. The NOP essentially models a 
sidp behavior. If the operation code corresponds to an 
NOP, a valid message is retumed in Step 336 and the 
process terminates in Step 342. However, if the opera- 
tion code is not an NOP, infomnation associated with the 
item is read from the change log 'ID, Parent ID, flags, 
type' in Step 321 . 

[0039] In Step 324, the invention detemnines whether 
the operation code corresponds to a delete function. If 
the operation code corresponds to a delete function then 
a valid message is retumed in Step 336 and the process 
terminates in Step 342. However if the operation code 
does not correspond to a delete function, then while 
there is another field for the current item the field infor- 
mation for the item is obtained an added to the item in 
Step 330. The for-loop that begins in Step 327 ends in 
Step 333 when there are no additional fields for the cur- 
rent item. Thereafter, a valid message is retumed in 
Step 336 and the process tenminates in Step 342. The 
following steps summarize the "Getltemlnfo" sequence 
of Fig. 3: 

1 . If read position past end return invalid; 

2. Read operation code; 

3. If it is invalid retum valid; 

4. Get current position; 

5. If operation is a 'NOP' (no operation) return valid; 

6. Read item Information; 

7. If operation is a delete retum valid; 

8. Get field infomnation and add to current item; 

ii. Repeat get rieid ana add Tor each tieid in the cur- 
rent item record; and 
10. Retum valid. 

[0040] Figure 4 shows an expanded view of Step 209 
of Figure 2. At this point all the fields in a given item have 
been retrieved from the change log. In Step 403 the in- 
vention determines whether the item is in the map. If the 
item is not in the map then, in Step 409, the item is in- 
serted into the map, th3 rtsm and field lucatiori irnoritis- 
tion is updated in Step 41 8 and the process terminates 



in Step 421 . However, if the item is in the map then, in 
Step 406, the invention detemnines whether the item re- 
trieved is a file. If the retrieved item is a file then, in Step 
412, a sequence of steps to execute and 'add file item 

5 to map' is perfomned in Step 41 2. Thereafter the item in 
the field location infomnation is updated and the process 
temiinates In Step 421 . However, if the retrieved item is 
not a file then the sequence of steps to "default add item 
lo map IS penormeo m &iep 41 o. Again, tne iiem ana 

10 field location information is updated and the process ter- 
minates In Step 421. 

[0041 ] Figure 5 shows an expanded view of Step 41 2 
of Rgure 4. In Step 503 the invention detemnines wheth- 
er the file has a new base version. If the file has a new 

IS base version then, in Step 515, the existing entries are 
removed from the map, the records in the aggregate log 
are marked obsolete, the file is inserted into the nnap 
and the process ends in Step 525. However, if the file 
does not have a new base version then the invention 

20 determines whether the file a change, a force change, 
or a move in Step 506. ff the file is either a change, a 
force change, or a move then, in Step 51 6 the file is add- 
ed to the map. Rle item changes must be applied serially 
and cannot be aggregated. This requires the use of a 

25 map that allows a "one to many" relationship between 
keys and items to maintain a "stack" of item records for 
a single item. The process then ends in Step 525. 
[0042] However, if the file item is not a change, a force 
change, or a move, then the invention detemnines 

30 whether the file item is a delete in Step 509. If it is a 
delete, then in Step 5 1 9 the existing entries are removed 
from the map, the records in the aggregate log are 
marked obsolete, the delete is inserted into the map. 
and the process ends in Step 525. If the file item does 

35 not have a new base version, is not a change a force 
change or a move and is not a delete, then by default in 
Step 512 it is an add. In Step 522, the existing entries 
are removed from the map the records in the aggregate 
log are mariced obsolete and the add is inserted into the 

40 map. Once this has been done, the process ends in Step 
525. 

[0043] Rgure 6 shows an expanded view of Step 41 5 
. of Rgure 4. In Step 603 the invention detennines wheth- 
er the item (in this case, the item is not a file) is a change, 

45 a force change, or a move. If the item is a change, a 
force change, or a move, then, in Step 61 2, the item from 
the new log is merged with the item in the aggregate 
role. However, if the item is not a change, a force 
change, or a move, then, in Step 606, the invention de- 

50 termines whether the item is an "add." If the item is an 
add, then in Step 615. the existing entries are removed 
from the map. the records in the aggregate role are 
marked as ot>solete, and this add Is inserted into the 
map. However, if the item is not an "add" then in Step 

55 609. by default the item is then a delete. Subsequently, 
in Step 618, the existing entries from the map are re- 
moveu, ti'ic- rc-coidi in uie at^grogaW log diO rriarke J as 
obsolete, and the delete is inserted into the map. After 
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Steps 612, 61 5 and 61 8 the process ends with Step 621 . 
[0044] Rgure 7 shows an expanded view of Step 612 
of Figure 6. Beginning with Step 703. for each field in 
the current item the following steps are perfonmed. In 
Step 706, the present invention detennines whether the 5 
current field is a body or a formatted body field. If the 
current field is not a body or a fonmatted body field then 
the invention determines, in Step 712, whether a cun-ent 

then the fieids are merged in outside process 71 7. How- io 
ever, if the current field does not already exist then, in 
Step 720, the current field is added to the item in the 
aggregate log. Turning back to Step 706, If the field is a 
body or a fomiatted body field then the invention next 
determines whether the field has already been purged is 
in Step 709. If ft has not been purged then, in Step 714 
the previous instance of either field. If any, are found and 
purged and the current field is subsequently added to 
the item in the aggregate log in Step 720. However, if 
the field has been purged then Step 709 proceeds di- 20 
rectly to Step 720 where the current field is added to the 
item in the aggregate log. The for-loop beginning with 
Step 703 ends with Step 723 when the process has cy- 
cled through each Field in the current item. The process 
then tenminates In Step 726. ^ 
[<K345] Figure 8 shows an expanded view of outside 
process 717 of Figure 7. In Step 803, the invention first 
determines whether a given field is a collection. If the 
field is a collection then, in Step 812. for each field In 
the current collection the following steps are performed. 30 
In Step 818, the Invention determines whether the cur- 
rent field already exists. If the current field already exists 
then outside process 823 is performed and the field In 
the cun-ent collection is merged into the aggregate log. 
If the cunrent field does not already exist then, in Step 35 
826 the cun-ent field is added to the list. Step 829 then 
repeats Steps 818, 823 and 826 until there are no addi- 
tional fields in the current collection. Going back to Step 
803, if the field is not a collection then the invention de- 
termines whether the field is a body or a formatted body 
field in Step 806. If the field is not a body or a formatted 
body field then, in Step 814. the field Information is re- 
placed. However, if the field is a body or a formatted 
body field then, in Step 809, the invention determines 
whether the rield has already been purged. If the field 45 
has not already been purged then, in Step 816, the pre- 
vious instance of either field, if any, is found and purged, 
ir tne neid has oeen purged then, in Step 820 the cun-ent 
field is added to the item in the aggregate log. After 
Steps 829, 81 4, and Step 820. the process ends in Step 50 
832. 

[0046] Figure 9 shows an expanded view of outside 
process 230 of Figure 2. Starting with Step 903. the in- 
vention first determines whether there are any obsolete 
records. If not, the process ends in Step 972. However, ss 
if obsolete records exist then, in Step 906, a seek to the 
start of Tcozrclz in the aggrc-siits !og for both read and 
write instances is performed. The for-loop initiated by 



Step 909 begins in Step 912 with the operation code for 
the read instance being read in. In step 921 , the inven- 
tion detennines whether the operation code read in Step 
912 is a no-op. If the operation is a no-op, the operation 
code is written in step 933. If after Step 933. the end of 
the aggregate log has been reached in Step 966. then 
the aggregate log size is reset and the aggregate log is 
flushed in Step 969 and thereafter the process ends in 
Siep »/^. 

[0047] However, if the operation in Step 921 is not a 
no-op then, in Step 91 8, item infonnation comprising ID. 
Parent ID, type, and flags is read. In Step 915. so long 
as the current Item Is obsolete, the following steps are 
performed. In Step 924 the read pointer is advanced to 
the next operation code. In Step 936 the operation code 
is read and in Step 945 the invention determines wheth- 
er the operation is a no-op. If the operation is not a no- 
op then, in Step 954, item information comprising ID, 
Parent ID, type, and flags Is read. However, if Step 945 
results in a no-op, then, in Step 960. the process loops 
back to Step 915 while the current item is obsolete. At 
the same time, a write operation is perfonmed in Step 
927 and in Step 930, the invention detenmines whether 
this operation code is associated with a no-op. 
[0048] If the operation code of Step 927 is associated 
with a no-op, then the process proceeds to Step 966 
which then loops back to Step 909 unless the end of the 
aggregate log has been reached. However, if the oper- 
ation code of Step 927 is not associated with a no-op 
then in Step 942. item info (ID. Parent ID, type and flaps) 
are written, tf the operation code of Step 927 corre- 
sponds to a delete then the process proceeds to Step 
966, which then loops back to Step 909 unless the end 
of the aggregate log has been reached. If the operation 
code of Step 927 is not a delete then while there is an- 
other field for the cun-ent item the following steps, be- 
ginning with Step 943, are perfonmed. In Step 948, the 
field is read and in Step 957 the field is written. Once 
there are no additional fields for the cun-ent Item the 
process proceeds to Step 966 and proceeds to Steps 
969 and 972 If the end of the aggregate log has been 
reached. 

[0049] Figure 10 shows a flow chart illustrating the 
process for getting field information. The field tag and 
the cunrent position are Initially read in Step 1003. In 
Step 1 01 5 the invention detennines whether the field is 
a collection. If the field is a collection then, in Step 1 021 , 
the collection field (count) is read and saved. Then, in 
Step 1 033, the field/count pair is added to the collection 
field stack. The process then temiinates in Step 1066. 
However, tuming back to Step 1015. if the field is not a 
collection then, in Step 1006 the invention detennines 
whether the field is a part of a collec^on. If the field is 
not part of a collection then, in Step 1048 the field is 
added to the current item's field list. TTie change log is 
then advanced to the next field in Step 1060 and the 
process lerminales di Slep 1O66. Turning back to Step 
1 006, if the field is part of a collection then, in Step 1 009, 
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the field is saved on a holding stack. Thereafter in Step 
1 012. the following steps are performed until the collec- 
tion field stack is empty. In Step 1018 the invention de- 
termines whether the current collection ts full. If the cur- 
rent collection Is not full then, in Step 1 024, the invention 
determines whether the holding stack is empty. If the 
holding stack is empty the aggregate log is advanced to 
the next field as provided in Step 1 060 and the process 

not empty then, in Step 1030, the field is popped from 
the holding stack. The field is then added to the cun-ent 
collection field's list and its counter is decremented in 
Step 1045. 

[0050] Turning back to Step 1018, if the current col- 
let^on is full then, in Step 1027, the cun-ent collection 
is added to the holding stack and removed from the col- 
lec^on field stack Then in Step 1036, the invention de- 
termines whether the collection field stack is empty. If 
the collection field stack is empty then, in Step 1 051 . the 
field entry is popped off the holding stack and added to 
the parent item's field list until there are no more entries 
in the holding stack, so long there is an entry in the hold- 
ing stack (i.e., if the condition established by the for-loop 
defined by Steps 1042 and 1057 is still valid), Turning 
back to Step 1 036, if the collection field stack is not emp- 
ty then, in Step 1 039, the invention determines whether 
the cun^nt collection is full. If the current collection is 
not full then, in Step 1054, the field is popped off the 
holding stack, added to the current collection field, and 
the collection count is decremented. If the current col- 
lec^on is full, then Step 1039 proceeds directly to Step 
1063 and Step 1063 will loop back to Step 1012 unless 
the collection field stack is empty. 
[0051 ] The foregoing description of preferred embod- 
iments of the present invention has been provided for 
the purposes of illustration and description. It is not in- 
tended to be exhaustive or to limit the Invention to the 
precise forms disclosed. Many modifk:ations and varia- 
tions will be apparent to practitioners skilled in this art. 
The descn*bed embodiments were chosen and de- 
scribed in order to best explain the principles of the in- 
vention and its practical application to thereby enable 
others skilled in the art to understand the Invention for 
various embodiments and with various modifications as 
are suited to the particular use contemplated. It is in- 
tended that the scope of the invention be defined by the 
following claims and their equivalents. 



Claims 

1. A method for updating application data in a client- 
device of a data transfer and synchronization sys- 
tem, said method comprising the steps of: 

downloading a first change log of a plurality of 
c^an^c logs from a oervcjr systG-a'i, t>ac^l of said 
plurality of change logs reflecting changes to 



said application data; 

adding said first change tog to an aggregate 
log; 

delefing said first change log; 
5 repeating said downloading, adding, and delet- 

ing steps for a next change log of ssad plurality 
of change logs until no additional change togs 
exist; and 

cippiyiiiy uaiii uygregctie log to sato appitcaiion 
10 data to update said applk^ation data. 

2. The method of claim 1 . wherein said adding step 
further comprises the steps of: 

IS (a) retrieving information for a valid item in said 

application data; 

(b) updating a map of said aggregate log. said 
map storing meta-data; 

(c) writing said item to said aggregate log; 
20 (d) updating a location of said valid item in said 

map; and 

(e) repeating steps (a) - (d) for all remaining val- 
id Items of a current change log. 

25 3. The method of claim 2. further comprising the step 
of: 

(f) compacting said aggregate log if a compact 
threshold is exceeded. 

30 

4. The method of claim 1 , wherein said applfcation da- 
ta comprises data classes for contacts, intemet 
browser bookmarks, calendar events, email mes- 
sages, notes, tasks, and files. 

35 

5. The method of claim 4, wherein said contacts com- 
prise records identifying names, addresses, phone 
numbers, and email addresses for a plurality of in- 
dividuals. 

40 

6. The method of claim 4, wherein said files comprise 
word pnxessor specific documents, electronic 
presentations, spreadsheets, and executable files 
in binary format 

45 

7. The method of claim 1 , wherein said applk:ation da- 
ta is in a universal data forniat. 

8. An apparatus for updating applk:ation data in a cli- 
50 ent device of a data transfer and synchronization 

system, said apparatus comprising: 

a downloading routine for iteratively retrieving 
a plurality of change logs from a server system, 
55 each of said plurality of change logs reflecting 

changes to said applrcation data; 
a ir«ii9lriy roulirio for iterativoly agg.cgatirig ths; 
contents of said plurality of change logs to an 
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aggregate log; 

a change log deletion routine for iteratively de- 
leting said plurality of change logs; and 
an updating routine for applying the contents of 
said aggregate log to said applk^ation data to s 
update said application data. 

9. The apparatus of claim 8, wherein said merging rou- 

an Item retrieval routine for retrieving a plurality 
of items from said application data; 
a map update routine for updating a map of said 
aggregate log wherein said map stores meta- 
data; 

a field retrieval routine for retrieving field infor- 
mation from said plurality of Items from said ap- 
plication data; and 

an item location update routing for updating the 
location of a plurality of items in said map. 

1 0. The apparatus of claim 9, wherein said merging rou- 
tine further comprises: 

an aggregate log compacting routine. 

11. A method for aggregating the contents of accumu- 
lated change logs into an aggregate log and apply- 
ing said aggregate log to update application data In 
a first dient device of a data transfer and synchro- so 
nization system, said method comprising the steps 

of: 

downloading a first change log of a plurality of 
change logs from a sewer system, each of said 35 
plurality of change logs reflecting changes to 
said application data; 

adding said first change log to an aggregate 
log; 

deleting said first change log; ^ 
repeating said downloading, adding, and delet- 
ing steps for a next change log of said plurality 
of change logs until no additional change logs 
exist; and 

applying said aggregate log to said application 45 
data to update said application data. 
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